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DETAILED ACTION 

Response to Amendment 

1 . This is in response to an amendment/response filed on March 3 rd , 2010. 

2. Claims 1 and 22 have been amended. 

3. Claim 6 has been cancelled. Claims 3 and 8 were previously cancelled. 

4. No claims have been added. 

5. Claims 1, 2, 4-5, 7, and 9-29 are currently pending. 

Claim Rejections - 35 USC §103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

8. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
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claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S. C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

9. Claims 1, 2, 4, 6, 7, and 9-29 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Saito (U.S. Patent 6,987,767) in view of Maclnnis et al. (U.S. Patent 6,853,385). 

For claim 1, Saito discloses a method of forming an output media stream to be 
transmitted during a communication session from a portable communication device wherein said 
output media stream comprises signals of a first media type, the method comprising: 

generating in real time a first media stream in the portable communication device (see 
column 9 lines 14-24 and figure 4, which recite microphone 3 land camera 33 that generate a 
real-time media stream as input to multiplexer 22), 

combining in real time the first media stream with a second media stream to form the 
output media stream (see column 9 lines 62-67, column 10 lines 1-2, and figure 1, which recite a 
multiplexer 22 that combines a media stream generated by microphone 31 and camera 33 with 
media streams of various other encoding units), 

wherein combining comprises combining signals of the first media type from the first 
media stream with signals of the first media type from the second media stream (see column 4 
lines 20-27, 50-56, and figure 1, which recite combining a plurality of video streams and further 
combining a plurality of audio streams using the TS multiplexing section 310), 
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transmitting said output media stream (see column 9 lines 25-31, which recite 
transmitting the multiplexed stream using transmission circuit TX 15), 

wherein at least one of generating and/or combining is dependent on input data from a 
user of said portable communication device (see column 9 lines 14-31 and figure 4, which recite 
generating and combining media streams that depend on input data from the user through the 
microphone 31 and camera 33). 

Saito discloses all the subject matter of the claimed invention with the exception wherein 
combining media streams specifically comprises superposing signals of the first media type from 
the first media stream with signals of the first media type from the second media stream to 
produce the output media stream, wherein the output media stream comprises portions of the first 
and second media streams which are configured to be presented in a substantially simultaneous 
time. However, Maclnnis et al. from the same or similar fields of endeavor disclose a system for 
combining audio, graphics, and video transport streams into blended output streams (see column 
1 lines 60-67, column 2 lines 1-16, and column 13 lines 5-16). Graphics that are blended may 
contain video (see column 1 lines 51-57) and multiple audio streams may be mixed in a manner 
similar to blended graphics (see column 7 lines 61-67, and column 8 lines 1-7). Thus, it would 
have been obvious to the person of ordinary skill in the art at the time of the invention to use the 
system that superposes media streams as taught by Maclnnis et al. with the communication 
device configured to multiplex media streams to form an output media stream as taught by Saito. 
The multiplexed MPEG transport streams can be superimposed into an output stream by 
implementing the arrangement of multiplexers for superposing media streams as taught by 
Maclnnis et al. as the multiplexing section 3 1 0 as taught by Saito et al. The motivation for using 
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the arrangement of multiplexers for superposing media streams as taught by Maclnnis et al. as 
the multiplexing section 310 of the communication device configured to multiplex media streams 
to form an output media stream as taught by Saito is to reduce the cost of the system by 
integration of hardware functions, efficient use of memory, and efficient utilization of CPU 
activity (see Maclnnis et al, column 2 lines 42-60). 

For claim 2, Saito discloses disclose a method for forming an output media stream 
wherein said output media stream comprises signals of a second media type (see column 9 lines 
25-31, which recite a multiplexed transport stream containing both audio and video data). 

For claim 4, Saito discloses a method of forming an output media stream that further 
comprises establishing a connection with another device (see column 8 lines 23-50 and column 9 
lines 14-31, which recite a transmission circuit 15 used to establish a connection with another 
device). 

For claim 7, Saito discloses disclose a method for forming an output media stream 
wherein combining comprises combining signals of the first media type from the first media 
stream with signals of a second media type from the second media stream (see column 4 lines 
20-27 and 50-56, which recite combining the video streams of multiple encoding units and 
further combining the audio streams of the multiple encoding units using TS multiplexing section 
310). 

For claim 9, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises combining signals of a second media type from the first 
media stream with the signals from the second media stream (see column 4 lines 20-27 and 50- 
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56, which recite combining the video streams of multiple encoding units and further combining 
the audio streams of the multiple encoding units using TS multiplexing section 310). 

For claim 10, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises combining signals from the first media stream with signals 
of the second media type from the second media stream (see column 4 lines 20-27 and 50-56, 
which recite combining the video streams of multiple encoding units and further combining the 
audio streams of the multiple encoding units using TS multiplexing section 310). 

For claim 11, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises combining signals of the second media type from the first 
media stream with signals from the second media stream (see column 4 lines 20-27 and 50-56, 
which recite combining the video streams of multiple encoding units and further combining the 
audio streams of the multiple encoding units using TS multiplexing section 310). 

For claim 12, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises delaying, prior to combining, signals of one media type of 
the second media stream in relation to the other media type of signals of the same stream to 
provide synchronized signals from the second media stream within the output media stream (see 
column 7 lines 23-31, which recite delaying signals of one media type by shifting the value of the 
Presentation Time Stamp PTS value to a later time to provide synchronized signals) . 

For claim 13, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises independently combining signals of the first media type 
and signals of the second media type (see column 4 lines 20-27 and 50-56, which recite 
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combining the video streams of multiple encoding units and further combining the audio streams 
of the multiple encoding units using TS multiplexing section 310). 

For claim 14, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises delaying signals of one media type within the output media 
stream, in relation to the other media type of signals of the same stream to provide synchronized 
signals from the first media stream within the output media stream (see column 7 lines 23-31, 
which recite delaying signals of one media type that is part of the multiplexed transport stream 
by shifting the value of the Presentation Time Stamp PTS value to a later time to provide 
synchronized signals). 

For claim 15, Saito discloses disclose a method for forming an output media stream 
wherein the signals of the first media type are audio signals so that the signals of the first media 
type from the first media stream comprise first audio signals and the signals of the first media 
type from the second media stream comprise second audio signals (see column 4 lines 20-27 and 
50-56, which recite combining the video streams of the encoding units and further combining the 
audio streams of the encoding un its using th e TS multiplexing section 310), wherein combining 
further comprises superposing the first and second audio signals of the first and second media 
streams (see column 4 lines 57-67 and figure 6, which recite superposing the audio streams into 
a single transport stream by multiplexing individual partitioned PES portions of the parse-PES 
generation sections 612 to 6n2 into a single transport stream) . 

For claim 16, Saito discloses disclose a method for forming an output media stream 
wherein superposing comprises weighting properties of the audio signals from the first media 
stream and the second media stream (see column 4 lines 57-67 and figure 6, which recite 
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superposed audio streams with weighting properties based upon the amount of partitioned PES 
portions that each audio stream provide for multiplexing to the TS multiplexing section 310, 
since the more partitioned PES portions are provided for multiplexing, the more data from that 
stream becomes part of the multiplexed transport stream). 

For claim 17, Saito discloses disclose a method for forming an output media stream 
wherein the signals of the first media type are image signals so that the signals of the first media 
type from the first media stream comprise first image signals and the signals of the first media 
type from the second media stream comprise second image signals (see column 4 lines 20-27 and 
50-56, which recite combining the video streams of the encoding units and further combining the 
audio streams of the encoding units using the TS multiplexing section 310), wherein combining 
further comprises blending the first and second image signals of the first and second media 
streams (see column 4 lines 57-67 and figure 6, which recite superposing the video streams into 
a single transport stream by multiplexing individual partitioned PES portions of the parse-PES 
generation sections 611 to 6nl into a single transport stream) . 

For claim 18, Saito discloses disclose a method for forming an output media stream 
wherein blending comprises weighting properties of the image signals from the first media 
stream and the second media stream (see column 4 lines 57-67 and figure 6, which recite 
superposed videostreams with weighting properties based upon the amount of partitioned PES 
portions that each audio stream provide for multiplexing to the TS multiplexing section 310, 
since the more partitioned PES portions are provided for multiplexing, the more data from that 
stream becomes part of the multiplexed transport stream). 
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For claim 19, Saito discloses disclose a method for forming an output media stream 
wherein weighting properties includes varying the proportion of signals from the first media 
stream in relation to the proportion of signals from the second media stream (see column 4 lines 
57-67 and figure 6, which recite superposed audio streams with varying proportion of signals 
since as more partitioned PES portions are provided for multiplexing by the audio stream, less of 
the resulting multiplexed transport stream comprises audio from the other audio streams). 

For claim 20, Saito discloses disclose a method for forming an output media stream 
wherein weighting properties is dependent on input data of a user of said portable 
communication device (see figure 4, which recite microphone 31, camera 32, and key input 
section 35 of input section 3 which takes input data of a user, wherein the weighting properties 
of the input data depends on the amount of input data provided by the user). 

For claim 21, Saito discloses disclose a method for forming an output media stream 
wherein varying said proportions comprises varying of each proportion within the range between 
0 and 100% (see column 4 lines 57-67 and figure 6, which recite superposed audio streams with 
weighting properties based upon the amount of partitioned PES portions that each audio stream 
provide for multiplexing to the TS multiplexing section 310, wherein an audio stream may 
provide all of the audio data to be multiplex or no data to multiplex). 

For claim 22, Saito discloses a portable communication device configured to form an 
output media stream to be transmitted during a communication session from said portable 
communication device, wherein said output media stream comprises signals of a first media type, 
said portable communication device comprising: 
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at least one generating unit configured to generate a first media stream (see column 9 
lines 14-24 and figure 4, which recite microphone 3 land camera 33 that generate a real-time 
media stream as input to multiplexer 22), 

a first combining unit, connected to said generating unit, wherein the first combining unit 
is configured to combine in real time the first media stream with a second media stream (see 
column 9 lines 62-67, column 10 lines 1-2, and figure 1, which recite a multiplexer 22 that 
combines a media stream generated by microphone 31 and camera 33 with media streams of 
various other encoding units), 

wherein the first combining unit is configured to combine signals of the first media type 
from the first media stream with signals of the first media type from the second media stream to 
form the output media stream (see column 4 lines 20-27, 50-56, and figure 1, which recite 
combining a plurality of video streams and further combining a plurality of audio streams using 
the TS multiplexing section 310), 

a control unit configured to control the generating unit and the combining unit, in 
dependence of user input (see figure 4, which recite a main control portion 21 that receives user 
input from input section 35), 

a transmitter configured to transmit said output media stream (see column 9 lines 25-31, 
which recite transmitting the multiplexed stream using transmission circuit TX 15), 

wherein at least one of the generating unit and/or the combining unit is configured to 
function in response to input data from a user of said portable communication device (see 
column 9 lines 14-31 and figure 4, which recite generating and combining media streams in 
response to input data from the user through the microphone 31 and camera 33). 
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Saito discloses all the subject matter of the claimed invention with the exception wherein 
combining media streams specifically comprises superposing signals of the first media type from 
the first media stream with signals of the first media type from the second media stream to 
produce the output media stream, wherein the output media stream comprises portions of the first 
and second media streams which are configured to be presented in a substantially simultaneous 
time. However, Maclnnis et al. from the same or similar fields of endeavor disclose a system for 
combining audio, graphics, and video transport streams into blended output streams (see column 
1 lines 60-67, column 2 lines 1-16, and column 13 lines 5-16). Graphics that are blended may 
contain video (see column 1 lines 51-57) and multiple audio streams may be mixed in a manner 
similar to blended graphics (see column 7 lines 61-67, and column 8 lines 1-7). Thus, it would 
have been obvious to the person of ordinary skill in the art at the time of the invention to use the 
system that superposes media streams as taught by Maclnnis et al. with the communication 
device configured to multiplex media streams to form an output media stream as taught by Saito. 
The multiplexed MPEG transport streams can be superimposed into an output stream by 
implementing the arrangement of multiplexers for superposing media streams as taught by 
Maclnnis et al. as the multiplexing section 3 10 as taught by Saito et al. The motivation for using 
the arrangement of multiplexers for superposing media streams as taught by Maclnnis et al. as 
the multiplexing section 310 of the communication device configured to multiplex media streams 
to form an output media stream as taught by Saito is to reduce the cost of the system by 
integration of hardware functions, efficient use of memory, and efficient utilization of CPU 
activity (see Maclnnis et al, column 2 lines 42-60). 
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For claim 23, Saito discloses a portable communication device configured to form an 
output media stream, wherein the first combining unit is configured to combine signals of the 
first media type of both the first and the second media streams, wherein the output media stream 
comprises signals of the first media type and a second media type, wherein the portable device 
further comprises: 

a second combining unit configured to combine signals of the second media type of the 
first media stream and signals of the second media type of the second media stream by using the 
second combining unit (see column 4 lines 20-27 and 50-56, which recite TS multiplexing 
section 310 of multiplexer 22 that combines the video streams of the encoding units and further 
combines the audio streams of the encoding units). 

For claim 24, Saito discloses a portable communication device configured to form an 
output media stream, further comprising a memory unit configured to provide storage for the 
second media stream (see column 9 lines 1-4, which recite memory 26 that stores video and 
audio data). 

For claim 25, Saito discloses a portable communication device configured to form an 
output media stream, further comprising a user input interface configured to provide user input 
(see figure 4, which recite an input section 35 that provides input for the main control portion 21 
that is responsible for forming the multiplexed media streams). 

For claim 26, Saito discloses a portable communication device configured to form an 
output media stream wherein said device further comprises a multiplexing unit configured to 
provide synchronization of signals of one media type from the first media stream in relation to 
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signals of the other media type from the same first media stream, within the output media stream 
(see column 7 lines 23-31, which recite providing synchronization of the video and audio data). 

For claim 27, Saito discloses a portable communication device configured to form an 
output media stream, further comprising a delaying unit configured to provide synchronized 
signals within the output media stream (see column 7 lines 20-31, which recite delaying signals 
of one media type by shifting the value of the Presentation Time Stamp PTS value to a later time 
to provide synchronized signals video and audio signals). 

For claim 28, Saito discloses a portable communication device configured to form an 
output media stream wherein the delaying unit provides synchronization of signals from the 
second media stream, prior to combining with the first stream (see column 7 lines 20-31, which 
recite delaying signals of one media type by shifting the value of the Presentation Time Stamp 
PTS value to a later time to provide synchronized signals video and audio signals). 

For claim 29, Saito discloses a portable communication device configured to form an 
output media stream wherein the delaying unit provides synchronization of signals of one media 
type in relation to signals of the other media type from the same second media stream (see 
column 7 lines 20-31, which recite delaying signals of one media type by shifting the value of the 
Presentation Time Stamp PTS value to a later time to provide synchronized signals video and 
audio signals). 

10. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Saito (U.S. Patent 
6,987,767) in view of Maclnnis et al. (U.S. Patent 6,853,385) and further in view of Knuutila et 
al. (U.S. Patent 6,810,035). 
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For claim 5, Saito and Maclnnis et al. disclose all the subject matter of the claimed 
invention with the exception wherein the established connection is a circuit-switched connection. 
Knuutila et al. from the same or similar fields of endeavor disclose a wireless terminal that 
communicates real time media using a circuit switched connection (see column 2 lines 22-32). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the wireless terminal that establishes circuit-switched connections as taught by 
Knuutila et al. with the portable communication device configured to form an output media 
stream as taught by Saito. The wireless terminal that establishes circuit-switched connections 
can be implemented by using the radio transceiver 204 as taught by Knuutila et al. as the radio 
transceiver of the portable communication device as taught by Saito. The motivation as 
suggested by Knuutila et al. for using the wireless terminal that establishes circuit-switched 
connections with the portable communication device configured to form an output media stream 
is to provide communication of real time service (see column 2 lines 42-44). 



Response to Arguments 

11. Applicants' arguments with respect to claims 1, 2, 4-5, 7, and 9-29 have been considered 
but are moot in view of the new ground(s) of rejection. It is noted with appreciation that the 
Applicant has carefully considered the previous Office Action and cited prior art as evidenced by 
the Applicant's Remarks filed March 3 rd , 2010. 

In consideration of the Applicant's Remarks, the Applicants first assert, "the Office 
Action allegation that 'superimposing' would improve system performance appears to be 
unrelated to the recitations of Claim 1, which generally disclose new functionality in a portable 
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device" (see Applicants ' Remarks, page 8). Although the Applicants assert that improving the 
performance of the system is unrelated to the recitations of claim 1, it is noted that the limitations 
of claim 1 do not preclude improving the performance of the system. It is further noted that the 
system that superposes media streams as taught by Maclnnis et al. also reduces the cost of the 
system by the integration of hardware functions, efficient use of memory, and efficient 
utilization of CPU activity (see Maclnnis et al, column 2 lines 42-60), thus allowing a wider 
deployment of the portable media devices as taught by Saito et al. 

Second, the Applicants assert, "the 'blended graphics,' as described in Maclnnis, appears 
to refer to a blend of graphics images and video data. (See, e.g., Maclnnis, column 46, lines 57- 
58, FIG. 61 .) In contrast with video data, graphics display a static image that may be updated by 
another static image. For example, a graphic may include an active window in a window-driven 
operating system, among others. In this regard, the 'blend' is not of data of the same media type 
as recited in Claim 1. Thus, the blended graphics described in Maclnnis are not applicable to the 
recitations of Claim 1" (see Applicants ' Remarks, page 8). However, the Examiner respectfully 
disagrees with the Applicants' characterization of the Maclnnis reference. It is initially noted 
that the Maclnnis reference "includes references to both graphics and video, which reflects in 
certain ways the structure of the hardware itself. This split does not, however, imply the 
existence of any fundamental difference between graphics and video, and in fact much of the 
functionality is common to both. Graphics as used herein may include graphics, text and video" 
(see Maclnnis et al, column 1 lines 51-57). Since graphics as described by Maclnnis et al. may 
include video, blending graphics with video may be interpreted as blending data of the same 
media type as recited by claim 1 . Further, Maclnnis et al. explicitly describes a blending 
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multiple videos including video stored in a FIFO and passthrough video from a multiplexer (see 
column 13 lines 5-16). 

The Applicants additionally assert, "the picture-in-picture of Maclnnis appears to be 
described in the context of a 'data transport' for descrambling MPEG transport streams. (See, 
e.g., Maclnnis, column 72, lines 55-64.) In this regard, data transport as described in Maclnnis is 
wholly distinctive from portable communication devices of Claim 1 . For at least the above 
reasons, Maclnnis does not disclose or suggest what is alleged by the Office Action. 
Additionally, the Office Action does not allege and Saito does not disclose or suggest the 
teachings that are missing from Maclnnis" (see Applicants ' Remarks, page 8). However, it is 
noted that the rejection of the claim additionally cites the blending feature disclosed by Maclnnis 
et al, wherein blending the video and graphic streams is interpreted as superposing media 
streams as recited by claim 1 . 

The Applicants additionally assert, "Saito does not disclose or suggest 'wherein at least 
one of generating and/or combining is dependent on input data from a user of said portable 
communication device,' as recited in Claim 1, as amended" (see Applicants ' Remarks, page 9). 
However, it is noted that Saito discloses using user input data from microphone 3 1 and camera 
33 to generate a data stream that is combined with additional data streams to form an output 
stream (see Saito, column 4 lines 40-49, column 9 lines 14-31 and figure 1). Therefore, using the 
input data from a user to generate a media stream and combining the generated media stream 
with other media streams as taught by Saito is interpreted as generating and/or combining media 
streams that is dependent on input data from a user of said portable communication device as 
recited by claim 1 . 
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Finally, the Applicants assert, "Saito's discussion of packetizing the elementary streams 
and multiplexing the packets into a transport stream does not disclose or suggest superposing 
streams into one stream" (see Applicants ' Remarks, page 10). In response to applicant's 
arguments against the references individually, one cannot show nonobviousness by attacking 
references individually where the rejections are based on combinations of references. See In re 
Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 
USPQ 375 (Fed. Cir. 1986). Initially, it is noted that Saito discloses multiplexing multiple video 
streams with multiple audio streams for transmission as a MPEG stream (see Saito, column 4 
lines 9-16 and figure I). Additionally, Maclnnis et al. specifically disclose an arrangement of 
multiplexers to blend various media streams (see Maclnnis et al., column 1 lines 50-67, column 2 
lines 1-60, column 7 lines 61-67, and column 8 lines 1-7). Using the broadest reasonable 
interpretation, blending the media streams is interpreted as superposing the media streams. 
Therefore, the combination of Saito et al. and Maclnnis et al. disclose superposing streams into 
one stream as recited by claim 1 . 

For reasons similar to those provided above, the Applicants' remarks regarding 
independent claim 22 are not persuasive. The Applicant further argues that since the dependent 
claims depend on the argued independent claims, they are patentable by virtue of their 
dependencies. Since the Applicant's arguments regarding the independent claims are not 
persuasive, the applicant's arguments regarding the dependent claims are also not persuasive. 
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12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to BENH. LIU whose telephone number is (571)270-3118. The 
examiner can normally be reached on 9:00AM to 6:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ricky Ngo can be reached on (571)272-3 139. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Ricky Ngo/ 

Supervisory Patent Examiner, Art Unit 
2464 
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